A DBMS is a suite of computer programs that are designed to manage a database, which is a large set of structured data. In particular, a DBMS is designed to quickly access and analyze data on large amounts of stored data. Most modern DBMS systems comprise multiple computers (nodes). The nodes generally communicate via a network, which will use a network protocol, such as HTTP, or raw TCP/IP. Information that is exchanged between nodes is exchanged by packets, the specific format of which will be determined by the specific protocol used by the network. The data wrapped in the packet will generally be compressed to the greatest extent possible to preserve network bandwidth. Accordingly, when it has been received, it will have to be formatted for use by the receiving node. A variety of DBMSs and the underlying infrastructure to support them are well known in the art. Database input/output (“I/O”) systems comprise processes and threads that identify, read, and write blocks of data from storage; e.g., spinning magnetic disk drives, network storage, FLASH drives, or cloud storage.
Like many software systems, DBMS evolved from standalone computers, to sophisticated client/server setups, to cloud systems. An example of a cloud based DBMS is depicted in FIG. 1. In particular, a cloud system 2 will generally comprise a variety of nodes (computers) as well as software that operates on the nodes. The cloud system 2 will comprise numerous separate nodes, including multiple database servers 1. Each database server will maintain separate storage (not depicted), which will store some part of the maintained data. Various clients can access the cloud system 2 through the Internet 4. Clients can include, for example, a standard desktop or laptop computer 6, a mobile device 7, as well as various sensors 8 and control equipment 9.
Generally, DBMSs operate on computer systems (whether standalone, client/server, or cloud) that incorporate operating systems. Operating systems, which are usually designed to work across a wide variety of hardware, utilize device drivers to abstract the particular functions of hardware components, such as, for example, disk controllers, and network interface cards. As drivers are generally accessed through an operating system, such accesses will typically entail significant resource overhead such as a mode switch; i.e., a switch from executing application logic to operating system logic, or a context switch; i.e., the pausing of one task to perform another. Such switches are typically time consuming; sometimes on the order of milliseconds of processor time.
Data stored in a DBMS is usually stored redundantly, using, for example, a RAID controller, Storage Area Network (“SAN”) system, or dispersed data storage. For example, using prior art RAID techniques, data may be split into blocks and to ensure that data is recovered; and one or more nodes within a coding line can maintain parity. For example, for a sixteen node system, twelve nodes may store data blocks, while four nodes would store parity blocks. Other data/parity configurations, such as thirteen and three, or ten and six, could be used, and parity can be determined using any of the well-known prior art techniques.
A DBMS system must maintain a record of storage that has not yet been utilized. Slow speed databases can make use of the operating system to manage open storage. However, high speed databases need to maintain their own list of open storage. In addition, such databases need to utilize a method to determine when to recycle storage that had been in use, but is no longer in use.
A fundamental problem in distributed computing and multi-agent systems is to achieve overall system reliability in the presence of a number of faulty processes. This often requires processes to agree on some data value that is needed during computation. Examples of applications of consensus include whether to commit a transaction to a database, or agreeing on the identity of a leader. There are a number of such methods of agreement that are well known in the prior art.